Speakers will die earlier with air leaks than in a proper box.

Low power won't hurt them. they won't handle max power without being properly damped.

so the short answer is if you want to do 5-10 watt tests no worries.

You haven't described the fit problem so nobody can offer help with that.